Point of No Return: Suggesting An Absolute Age Cutoff for the Resuscitative Thoracotomy

Introduction

About me:

Jacob Canfield

• May/ may not have seen around over the summer
  

BS Human Physiology and BS Biochemistry & Molecular Biology (Dual Degree) - Michigan State University (2020)

My Previous Research (Proof I Have Experience Rriting Code and Doing Research):

Ultrafast femtosecond pulsed laser imaging of the retina (1st year of undergrad)

Multimodal nonlinear optical imaging of unstained retinas in the epi-direction with a sub-40 fs Yb-fiber laser

• Gabrielle A. Murashova, Christopher A. Mancuso, Jacob L. Canfield, Sanae Sakami, Krzysztof Palczewski, Grazyna Palczewska, and Marcos Dantus
• 3rd author

Creation of machine learning algorithm for predicting the gene expression of unmeasured genes (4 years, starting in undergrad)

A flexible, interpretable, and accurate approach for imputing the expression of unmeasured genes

• Christopher A. Mancuso*, Jacob L. Canfield*, Deepak Singla, Arjun Krishnan
• Co-first author
  

EMT-B (Chicagoland)

Starting MS-II in a week at Midwestern University

Summer Research Fellowship

Resucitative thoracotomy (RT) or Emergency Department Thoracotomy (EDT)

• Very high mortality rate
• Should there be an age limit (risk vs reward analysis) Two camps:
  1. RT is a last ditch effort, what do we have to lose
  2. RT incredibly low rate of success, has risk to the provider, resource intensive (especially at a smaller/ resource scarce hospital), can zap the blood bank, not a common procedure in general and as a result many practitioners do not have large volume of practice both performing and orchestrating team to perform this procedure efficiently and as effectively as possible
  • Both groups have valid points which is why robust guidelines are necessary to finely dictate when to pursue this avenue during a resuscitative effort.
  • An important aspect of these algorithms/ guidelines put for is that they are simple to use, use very little input information (speed is key) and most importantly should be evidence based.
    

We proposed investigating whether there is a substantial increase in mortality based on age and then determining an upper (and possibly lower) limit above (or below) which RT would be contraindicated.

• We of course recognize that there is no world in which we are making these big clinical decisions solely based off of age, however, including age in the guidelines would be useful for making an informed decision, allowing one to perhaps pursue other more viable avenues or implement maneuvers that have higher success rate or could buy time to make it to the OR where success rate is higher.
  

• Additionally, preserve resources, blood products and keep providers out of the way of unnecessary harm.

There is a paper that was done by Loyola in 2017 that aimed to pursue a very similar idea:

Inclusion criteria for data from NTDB:

• ICD: 34.02 - Exploratory Thoracotomy
• The above procedure was performed within the first 15 minutes of arrival to the ED
• The above procedure was performed in a number of minutes that was less than their time to discharge from the ED (i.e. anyone who got to the ED and was whisked away to the OR in the first 15 minutes and was, as a result, discharged from the ED, would be excluded)
• A listed mechanism of injury (MOI) that was either: Penetrating or Blunt
• A known age (if this data was missing, it was excluded)
• 10 year age bins were used for calculating mortality (they also did individual years)

Data from 2008-2012 from NTDB were used

• N = 2,585
• Overall Mortality Rate: 92.8%
• Penetrtating Mortality Rate: %92.3%
• Blunt Mortality Rate: 94.9%
• Mortality Rate >57 y/o: 99.4% (N= 140; There was a single individual >60 y/o who survived)
• Mortality Rate <11 y/o: 80% (N=20)

Foreshadowing from this paper:

A recent study investigating penetrating cardiac injuries in NTDB devel- oped a predictive model for outcomes with a predictive power of 93% that was more robust than its previous counterparts. Advances in machine learning, predictive analytics, and increased standardization of large databases such as the NTDB will allow us to better utilize evidence-based medicine for critical decision- making and improved patient outcomes.

Data Acquisition, Exploration and Analysis Methods

Data from the years 2007 - 2022 were requested and received from the NTDB

• Data in the .csv format was used for this project
• Data was uploaded using Python 3 along with various packages, namely pandas and numpy
• Data was explored year by year.
  • For each year there are several files
  • Data, each tied to a unique INC_KEY or Incident Key which is assigned to each patient
  • Description Files:
    • These are used to create interpretable Text out of the integer based labeling system used by certain categorical datafields within the data
  • Data Dictionary which can be used the same as the above (they have slightly different combinations and formats of information so you need both)
  • User Manuals (.pdf) that describes what a lot of the data fields are, what file they can be found in, what years they were actively used and if they are renamed from previous years so you can map this data back to those

This may not seem like that big of a deal but there were an immense number of inconsistencies, changes in file formats and a complete overhaul in the data and file structures that occur in 2016 - 2017 along with transition from ICD9 to ICD10, in which the word Thoracotomy does not exist (there is an entire paper in which some very upset indivudals complained about the implications of this).

• The variability in which different datafields is largely prior to 2016, with the exception of the massive changes that happen during 2017.
  

• I have spent no less than 60 hours on this task alone of this summer tinkering with the data files in this 2007 - 2016 data range to play nicely with anything from the 2017 - 2022. I've tried many methods to both automate and borderline manually combine this data with the newer data, however, for at least 10 large technical reasons and to little avail, I have not been successful in this endeavor (at least not entirely)
  

• For a subset of datafields that exist across all of the years of data (2007-2022) I can seemingly get it to mesh reasonably well with the newer data, HOWEVER, when I plot the mortality rates of the data in the 2007 - 2016 range, it is excessively low (~30-50%).
  

• Clearly there is something that is wrong with my filtering of the data that is not capturing all of the deceased cases or is including an excessive number of survivor casese that should realistically not be included in this calculation.
  

• Going forward, I will continue to attempt to tinker and resolve this problem as I think it would approximately double the sample size of what is ultimately used (at this time) for this project (~7-8k).

Long story short, using the NTDB is not as user friendly and ready to use out of the box as most probably expect.

So how the heck did we find the RT's if it's not in the ICD10 codes anymore?

• With a some help from Dr. Brigode's ICD dictionary tool, a website that has a similar function (https://www.icd10data.com/), and helpful input for ideas to look for from Dr. Schlanser and Julian Henderson, we tracked down and isolated a list of ICD10 procedures that in the context of the ED and being performed within the first 15-20 minutes of arrival at the ED, would presumably only be done in the context of an RT (and also excluded all sternotomy procedures) were able to isolate a lot of patient's who likely underwent RT.
  

• In addition, later on I discovered that starting in 2013 there were a new subset of fields that had been sneakily introduced that included Hemorrhage Control Surgery along with the times following arrival at which they were subsequently performed.
  

• Thankfully, these do include the procedure Thoracotomy, as well as Sternotomy (exlcuded) amongst other damage control surgeries. The listed procedure is supposed to be the initial procedure performed, so if Thoracotomy is listed, this does not rule out that a Sternotomy was not done but it implies that it was not the first procedure or the primary one.
  

• Aside from the initial damage control procedure listed, there are no additional fields detailing if and what any subsequent damage control procedures were performed (i.e. if another damage control procedure was done first and then a thoracotomy was done after this, we would not know that information and this patient would be excluded)

• Ensuring not to double count individuals who had both the Hemorrhage Control Surgery labels and relevant ICD10 codes and only counting individuals who had multiple ICD10 codes once, I obtained a unique list of INC_KEY, grabbed all of the patient data on them from 2017 - 2022.
  

• For patients who did not have Thoracotomy as their damage control procedure but did have ICD10 procedures that were suggestive of a RT, I selected these patients (many of which had multiple of these procedures which is not surprising; getting intracardiac epi and a left visual inspection of the heart, open approach makes sense) and then selected the minimum time out of their RT related procedures as the time to RT.

Important Note:

There are instances in which for these cases had another damage control procedure done (i.e. sternotomy, laparotomy etc.) with a time to when that procedure was started and sometimes this damage control surgery took place prior to these procedures we are assuming are the result of a thoracotomy.

My point:

Is that there may have been damage control procedure other than RT performed prior to the RT but I am overriding the time stamp to reflect when we are assuming the RT was started as opposed to when any damage control started.

• I ensured to exclude anybody who had a Sternotomy as indicated by ICD10 codes or by the damage control label.

A point I would like to receive feedback on is whether it is best to:

Strictly use the cases where it is clearly labeled a Thoracotomy being used as a damage control procedure and exclude all other cases from the study (maintain the purity of the data at the cost of sample size)

OR

Include all of the ICD10 code Thoracotomy related procecures but that are not explicitly labeled a Thoracotomy by the damage control surgery label (current implementation, more samples but possibly lower purity).

The reason I want to know is because mixing in the ICD10 code data makes processing more complicated (although once it's done you don't have to think about it again if done correctly in the initial stages of the project, which I think I have done but again, more places to make mistakes with this route).

• You are also adding the possibility that the individual did not have a Thoracotomy done at all into the realm of possibility, although for these individuals certainly can't be excluded.

The difference in N is ~7k vs ~8.4k, so not a value that is insignificant but also not a magnitude change in case number (it's never black and white).

• Then I limited the data to these procedures being performed in the first 20 minutes from arrival at the ED - For a subset of years they started listing the time to procedure in Hours instead of Minutes and as a result these cases have times that were converted to minutes by me prior to filtering out by time
• Made sure that the procedure time occurred prior to discharge from the ED.

Results

Using Data ONLY labeled as RT by HMRRHGCTRLSURGTYPE

Using Data From ICD10 and HMRRHGCTRLSURGTYPE

My preference

Although I think that we would get the extra 1.3k cases by including the cases in which RT was likely based on ICD10 codes alone, I think maintaining the purity of the data would be better since such small swings in mortality % dramatically changes how you might interpet these figures.

Feedback on this point is of great importance

Caution: This is only using 2017-2018 data because they overhauled the way they stored Comorbid Conditions (the same applies for the next graph showing Hospical Complications)

Very frustrating reoccurring theme with the NTDB

These results are just weird

Discussion

1. From our data here, although we need more data for some of the extremes of age, can likely say that for those over the age of 70, RT is not typically survivable.
   

2. In addition, those who survive and have blunt MOI vs penetrating and undergo RT have substantially longer recovery roads ahead of them as well as longer time on ventilators and ICU stays

1. Those who do not survive, although not exclusively, typically have worse vital signs in the pre-hospital and initial presentation setting. Of note, the lines we have set in trauma as being critical threshold for vital signs seem to make clinical sense with the data presented here.
   

2. Of the surviving individuals, they did not have their RT until around 9-10 minutes after arrival whereas the deceased had their Rt within 5-6 minutes, likely indicating the severity of their injuries. The surviving group likely has more time to stabilize patient in other ways prior to initiating RT or patient arrives in better condition leading to overall better outcomes (hypothesis).

1. Those who survived received a larger volume of blood products in the first 4 and 24 hours compared to the decased, however, this is likely skewed simply due to the fact that the vast majority of patients do not survive the first 30 minutes after arrival and thus are not transfused these larger volumes of products.
   

2. With the exception of outliers, all of the deceased had a initial ED GCS of 3, whereas those that survived had a range of 3-15.

1. Comorbid conditions did not seem to yield a lot of meaningful information given that the trauma population is largely young and reasonably healthy in addition to the lack of information that was able to be obtained from the pt/EMS due to patient condition or lack of records.
   

2. Alcohol and recreational drugs did not seem to provide any type of survival advantage and it seems typically was associated with worse outcomes.

Conclusions

1. At the age extremes, I think we need a lot more data to more definitive guidelines but I think this is still important and informative information
   

2. Part of fixing the above is getting the older NTDB to work with the newer stuff as this would at least 2x the samples available to study (some of the side pieces of information is not present in the older data so secondary analysis may be tricker)

1. Propose the addition of Transport time and minutes of CPR prior to arrival as new data fields that should be included in NTDB (would make it possible to evaluate the quality of decision algorithms for RT made by EAST/WTA going forward to see if those who do/do not meet those criteria and undergo RT have better or worse outcomes) but need these two data points to do that well.
   

2. Request that vital signs data be put back into NTDB instead of having to get a NEMESIS ID from the NTDB and then go try and retrieve it from NEMESIS.... (starting in 2021 EMS vitals are no longer tracked in the NTDB).

Future Directions

1. Would like to investigate the outcomes based on the subprocedures being performed (i.e. intracardiac epinephrine, cross-clamping the aorta etc.) to see if there are differences in outcomes when these are implemented (are these helping or hurting, or is it a matter of the severity of the patients injury that the mortality speaks about more than it does the procedure efficacy)
   

2. It would be interesting to see how far back this NEMESIS database goes back and see if it does happen to include timing data on CPR in the pre-hospital setting to then look at outcomes of patients who received RT in compliance or not with the guidelines put forth by WTA/ EAST
   

3. Investigate and compare sternotomy to RT that is performed on one or both sides and see if there are any major differences

Questions & Feedback

• What are the things about RT that you are personally curious about, think should be investigated or you think are areas of potential improvement
  

• Would love some feedback on if there is a feel for which statistical tests should be used to determine significance for a lot of this data, it seems like there are a lot of options, including those used in the Loyola paper, but I imagine it depends on how deep the weeds you want to get.

New plots I want to make in the future

1. Mortality vs. Numerical outcomes, MOI
2. Graph of LOS, ICULOS, Vent Days vs. age groups (Make one for using survior data and one using deceased data)
3. Graph showing outcomes separated by ED vs HOSP (x axis = ED and HOSP, y-axis = percentage; for the deceased, where did most of them die)
4. Plot showing what years different pieces of information/ datafields come and go
5. Try to look at something with severity scores and mortality (Create a heatmap based on these codes laid on top of human body and light up the most lethal areas)
6. Get HC and CC from 2019-2022 (remember it's in that weird format)
7. Demographics by age figures but instead of age use years to see trends

RTcotomy Machine Learning Tool

Go ahead and scan the QR code to go to the app!

Introduction

Purpose

• Make a machine learning algorithm that predicts whether an individual will survive a RT based on vital signs and basic information about the individual that you would have available to you prior to arrival or very quickly after arrival.
  

• Similar to the idea of a MedCalc tool (for example sepsis/ SIRS criteria)
  

• Although less pertinent at Cook County, a smaller community hospital who doesn't regularly perform RT may need PTA to mobilize appropriate resources, prepare team and staff for arrival of patient etc.
  

• Knowing upfront before arrival that you may or may not go down the avenue of using RT could be useful knowledge to be armed with
  

• Once you have their initial vital signs you could potentially use this to determine after the RT if they were expected to survive or not as a form of quality improvement (if the model thinks they should have been able to tolerate the RT, were there things that could have been done better, prepared for etc in these cases. Not pointing fingers or necessarily implying it was anything other than more specifics about the circumstances that can be captured by the model)

What is Machine Learning?

Back in Time

• Think back to organic chemistry (maybe don't do that) and remember Beer's Law in which you would create a serially diluted standard solution with known concentrations and you would use that with your spectrometer to fit a line for predicting the concentrations of an unknown solution.
  

• When you fit that line you did a form of regression (almost there!)

Linear equation

Your equation was something like this: y = mx + b

Logistic Regression Classification

• With machine learning it is the same thing except instead of their being only 1 dimension (x), there are many:
  

y = a1b1 + a2b2 + a3b3 ..... anbn + c where an is a weight (very much like m from above) times the value of a feature, b1.

• So basically the machine part of machine learning is whatever algorithm you are using to learn relationships about your data to tune your weights (an) so that the sum of all of these paired terms and the intercept, c, equal a prediction.
  

(This is incredibly reductionist, it's much more painful to learn than this, I don't want to go there and neither do you (but we could) but it is beyond the scope of right now)

In the case of my tool here the equation might look like this:

• Survived/deceased? = (a1 x EMS SBP) + (a2 x EMS Pulse Rate) + (a3 x EMS Respiratory Rate) + (a4 x EMS Total GCS) + c,
  

  where the values an are your weights and c is an intercept determined by the algorithm to fit your data.
  

• This is linear regression, and if you add a penalty to the weights that dictates how big or small they are allowed to be or how many of them will be forced to equal 0, you would have algorithms called RIDGE, LASSO, or ElasticNet (this one is basically a combination of both RIDGE AND LASSO and you tune a value to determine how much of each you want).
  • I use all 3 of these in my application, they're very simple but powerful algorithms, the paper I published in undergrad used LASSO
    

XGBoost Classification

• Lastly, there is very well studied, popular algorithm called XGBoost, which I have used in the app as well and it performs the best out of all of these.
  

  • If your head hurt from the last one, this one is no less than 10x more confusing, I will not even bother getting into the weeds with this one (I barely get it myself).

Methods & Results

v0.0

• v0.0 is pretty straightforward, you input all of the EMS data after selecting which algorithm you want to use and click Predict at the bottom and it will make a prediction along with returning the Confidence which is the probability with which the algorithm thought the outcome was the right answer.
  

• Basically the model assigns Deceased a value of 1, Survived a value of 0, and returns a number that is a percentage of how likely it thinks each class is to be the correct answer.
  

For example, a patient who sustained blunt trauma, is a 25 y/o male and had initial EMS vital signs of 0 SBP, 0 pulse rate, 0 respiratory rate and GCS of 3, the algorithm will return a value of 0.11 for the class 0 (Survived) and 0.89 for the class 1 (Deceased).

Then it selects the higher of these two values to make it's decision. The value 0.89 converted to a perctage is the Confidence that is the output along with the predicted class.

• v0.0 is simple and effective but has some drawbacks.
  • Does not include the ability to add initial ED vital signs
  • Does not tolerate missing information (i.e. if EMS doesn't give you an age or sex, you're out of luck)

v1.0

• In the meantime, I am still working through the kinks of v1.0 which allows you to select whether you want to use information from EMS, ED or Both settings
  

• Better yet, you can input any combination of at least 2 pieces of information or more and get a prediction (be mindful that obviously the less information given the less reliable your prediction will likely be, although this may not necessarily be reflected by the Confidence, I would like to find a better metric to use that incorporates the number of input datafields as a parameter)
  

• In order to do this you have to train a different machine learning model for every single combination of algorithm, input data fields, other parameters the machine learning algorithms use for training, also called tuning the hyperparameters
  

• Yes that's right, not only do you have different models, but each model has several variables where you have to make a combination for each of these hyperparameters, train the models on a subset of the data (which can not have missing data in it) to find the best hyperparameters, and then you take those and evaluate the model on one last held out group of data that the algorithm hasn't seen before.
  

• Also, if you were wondering how do you prevent your algorithm from becoming biased towards just predicting that everyone who gets a RT to be deceased since >90% of the time that would be a good assumption? You artificially try to make there be a approximately equal number of decased and survived patients in your training data so that you shield your algorithm from this bias.
  

In this case, I am also making a Dummy algorithm that is going to do exactly that: predict that the outcome is death no matter what the input is, and I will use this as a baseline for which my models should have to perform better than to determine that they are actually any good (as well as being better than random guessing or an AUROC >0.5)

You may be wondering, well what if the input data isn't a number and is instead categorical (i.e. Mechanism of Injury); you just convert each unique instance of categorical variable to an integer and then save a dictionary with these mappings that you use to translate back their meaning on the back end if you need to.

Ok so there's all of this machine learning stuff and then you have to now train 285k different models to make this application work (I used to have access to a super computer (MSU HPCC) which was basically the equivalent of 50k desktops put together that you could command at will to do massive amounts of code.

I am unfortunately limited to my MacBookPro (it ran for ~50.5 hours and churned out around 40k of the models, during which time I thought it might combust) and the desktop that I built a few years ago (~8X computing power).

v2.0

So this is where v2.0 comes in, my PC was needed so that my laptop wasn't running this code for 10+ days in a row. Turns out MacOS and Windows do not play nice when sharing your code between the two systems (I've never done before and will hopefully never do again), so the v2.0 is still in progress and running the models as we speak.

Additionally, there is a bug that I have not determined the cause of yet in v1.0 that is predicting the same example patient I presented earlier as having a 99% chance of survival (we all know that's not how that one is going to pan out, sorry kid) so I need to figure that out, although by the time v2.0 finishes, this may not be necessary at all.

The last but a certainly not least point that is to be made is that currently I am paying a $10 per month service to be able to host the website, however, this is only viable in the short term as it is still basicall being hosted from the Terminal in my laptop, so if my laptop is not running the code for the app, then the app does not exist online (problem)

In the future I will need to find a way to host this on a cloud server (something like Amazon Web Services (AWS)) that way it can be up all the time

Discussion

There's not a lot here yet as I still need to cultivate some effective ways to objectively measure how good the algorithms are

Need to be able to host the app without my laptop running around the clock like a guinea pig on a wheel

There could be a potential ability to allow other people to submit the outcomes of RT and create a database just with that purpose in my mind that could be used to improve the models (obviously de-identified and all the other hoops)

There are probably better methods out there that are more complex and advanced, and there is a lot I can do to mine right now to improve them without switching algorithms as an alternative avenue first

A concern is that this is all quite technical and although I have some small experience, having more experienced eyes on a project like this would probably be good so if this were to become an academic pursuit that would likely need to be an important consideration at some time in its development.

Questions & Feedback

• To make this a tool that you would use yourself, what features would you want to see added?
• If this is not something you could see yourself using but there was a similar tool/ use case that you would want to see that is similar to this, what would it be?
• Any other thoughts?

Gratitude

Immense thanks are owed to the many residents, attendings, nurses, other staff and even other medical students who mentored me this summer and let me feel like I was apart of the team!

Even at incredibly late hours of the night with never ending questions :)

I look forward to continuing to do Trauma/ Acute Care/ Critical Care research here

This is hands down the most positive experience I have had in my young medical career to date